
A Publication of the Institute for the Advancement of Emerging Technologies in Education at AEL 



Universal Design for Assessment 

An e-mail-based Soapbox discussion by IAETE, March 2005 



“ Perhaps the mandates that currently 
face education from state and federal 
levels should be the catalyst to start 
over and create from the beginning 
universally designed tools to evaluate 
student understanding, skills, and 
knowledge. ” 

-Tracey Hall 



Universal design is a process that ensures that a newly created product is useful 
to the broadest possible array of people, particularly those with disabilities. 
Often, designing for specific disabilities proves beneficial for all. The sloped 
curb corner, for example, allows wheelchairs, strollers, bicycles, and, indeed, 
any pedestrian to move more safely from the sidewalk to the street crossing. 

In March of 2005, the Institute for the Advancement of Emerging Technologies 
in Education (IAETE) at AEL held an online panel discussion on universal 
design for technology-based assessment as part of its ongoing Soapbox series. 



Participating in this weeklong, e-mail-based discussion were 



• Tracey Hall, Ph.D., Senior Research Scientist and Instructional De- 
signer at the Center for Applied Special Technology (CAST) 

• Nathan Sparks, Assessment Specialist at the Division of Assessment and 
Reporting, Virginia Department of Education 

• Irene Spero, Vice President of the Consortium for School Networking 
(CoSN) 

• Lynda Van Kuren, Communications Director for the Council for 
Exceptional Children (CEC). 



Applied to assessment, agree the panelists, universal design would mean that all 
students would have a better opportunity to express what they know by using 
their preferred methods for receiving and expressing information. 

This leveled playing field is not the current default for assessment design. 
Tracey Hall of the Center for Applied Special Technology (CAST) voices a 
view shared by all panelists when she writes, “The accuracy of current large- 
scale assessments is undermined by construct-irrelevant factors, including access 
barriers, a particular problem for students with disabilities.” 

However, greater access is an underlying tenet of universal design. As Irene 
Spero of the Consortium for School Networking points out, “Universal design 
in assessment is a dynamic and evolving issue.” Hall concurs, saying that 
research on the topic is “in its infancy.” 
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Indeed, it is difficult at this point to even define the role 
of accommodations in universal design of assessment. An 
often-used definition for the general concept is the design 
of products that will be usable by all people, to the 
greatest extent possible, with minimal need for additional 
adaptations and accommodations. For learning and 
assessment materials, however, Spero writes that “there is 
a consensus here that universal design must continue to 
recognize the need for appropriate accommodations.” 

It is clear that as states increase assessment activity, and 
as those assessments are created for technology-based 
environments, there is an opportunity to regularly 
involve and refine universal design in the assessment 
process. There is, say these experts, an opportunity to go 
beyond common question formats, such as multiple 
choice, and to offer students flexibility in how they 
demonstrate mastery of the standards. 

In brief, panelists are asked why retrofitting assessments 
is more common than beginning with universal design, 
what an assessment might look like when the principles 
of universal design are applied, what some of the chal- 
lenges are, what kind of professional development is 
needed, and where the relationship between accommoda- 
tions and universal design stands. 

Professional Development 

“The issue of universal design in assessment should not 
be confined to the special education community but 
must be disseminated to the larger education constitu- 
encies within schools and school districts. ” 

-Irene Spero 

Online assessment offers the potential for quick turn- 
around of data that can be used to immediately impact 
instruction. Unfortunately, some states leading the way 
in this innovation were not meeting the needs of stu- 
dents with disabilities. They are now retrofitting their 
online assessments for better access for these students. 

For example, one state originally presented text on 
screen as a graphic, in order to provide a higher level of 
security than text-based questions offer. Flowever, it was 
found that the graphics could not be read by text readers 
or clearly enlarged for students with visual disabilities. 

Why are states retrofitting for disability access rather 
than considering these needs from the outset? Awareness, 
time, and money, agree the panelists. But other forces are 
at work as well. Van Kuren of the Council for Excep- 
tional Children (CEC) points to person power: 

Even though awareness of universally designed 
assessments is growing, only a relatively few 



individuals know how to create assessments using 
universal design. The issue goes all the way back to 
ensuring the questions asked are truly measuring 
what is being assessed. 

Similarly, Sparks observes that the process can require 
“expertise outside of the state’s current development 
coordinators.” All panelists mention a need for far- 
reaching professional development. Assessment people 
need to learn more about universal design and the needs 
of students with disabilities. Teachers need to learn about 
universal design and assessment, including, in Spero’s 
words, “the basic elements of psychometric theory, 
validity, reliability, etc.” School teams that create and 
implement Individual Education Plans (IEPs) for stu- 
dents with disabilities must continually update their 
knowledge of changing accommodations and alternative 
assessments. 

Acknowledging that, as Spero writes, “professional 
development opportunities of this order will be complex 
and costly,” the group supports a train-the-trainer model 
for distributing information and establishing local 
experts. Virginia’s model of creating community experts 
particularly catches their attention. 

Flexibility: What Universal Design in 
Assessment Looks Like 

“The question becomes whether we want to play a 
numbers game or truly help students achieve. Already 
we are seeing that one-size-fits-all, high-stakes 
assessments are not working.” 

-Lynda Van Kuren 

All assessment begins with a clear definition of the 
construct being measured, and universal design of 
assessment begins here as well. From here, agree panel- 
ists, it immediately branches out— not necessarily to a 
novel test format but to flexibility in how students 
express their mastery of standards. The panel hopes not 
so much for assessment vehicles accessible via accommo- 
dations but for assessment systems that allow for a 
variety of learning styles and abilities. “A one-size-fits-all 
model is not effective for students with varied ways of 
receiving, processing, and expressing information,” 
explains Spero. Hall adds, 

Keeping in mind what we do know about univer- 
sal design, no one structure is “best.” Individual 
differences, abilities, and disabilities call for 
assessment and instructional design that is true to 
the construct being measured, and offers the 
flexibility to access and express in the mode or 
methods that best suits the individual. 
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While multiple-choice formats can ease test scoring and 
reporting, some characteristics of multiple-choice testing 
may inhibit or distract students with disabilities. Some 
head injuries, for example, can inhibit decision making; 
thus, a student who could fill in a blank on a reading 
item might have more difficulty responding to a mul- 
tiple-choice item. Others may be unable to manage a 
process of elimination, in which case the item evaluates 
reasoning rather than reading. Sparks observes that even 
with the best use of accommodations, certain assessment 
formats are simply not usable by some students: 

If we truly embrace some essential principles of 
universal design, we have to accept that the 
standard multiple-choice format (on which many 
state tests are based) may not provide complete 
universal access for all students. 

Many states have made efforts to incorporate plain 
language, assistive technology, more clearly 
defined constructs and measures, legibility in 
online structuring, and other universal design 
principles into current online assessment systems. 

This means that for test developers, many of these 
issues have been raised and considered on the top 
end with item writers and graphic artists. For 
many states, universal design has been discussed 
and implemented at many stages of the assessment 
development continuum. However, for those 
students needing alternatives to multiple-choice 
formats or extensive multisensory based items, 
these efforts may still fall short of providing 
universal design. 

“I believe the power of the IEP team to structure a 
student’s program should extend to assisting in structur- 
ing how a student demonstrates his or her knowledge,” 
adds Sparks. He suggests that, rather than responding to 
a series of multiple-choice questions to demonstrate 
knowledge of, say, President John F. Kennedy, perhaps a 
student could discuss Kennedy at length, incorporating 
essential knowledge and understandings. Given a fic- 
tional scenario with an unlimited budget, Sparks ob- 
serves that the discussion could be digitally captured and 
sent via webcast to a scoring company. 

Van Kuren identifies alternative assessments based on 
standards as an early form of the desired flexibility: 

States are developing alternative tests or giving out- 
of-level tests to some students and developing 
alternative high school diplomas to meet the needs 
of students who don’t fit the standard mold. As 
Dan Reschly, chair of the department of special 
education at Vanderbilt, said, “Administering tests 
to students who can’t answer any of the items is a 
waste of time as well as inhumane.” 



Questions of Validity and Reliability 

“It is often difficult to explain and justify why 
universal design principles will help all students and 
still maintain levels of validity, reliability, and item 
constructs. ” 

-Nathan Sparks 

Taken to an extreme, such flexibility could create a 
situation in which every student considers every possible 
alternative or accommodation for assessment. For 
example, an average student might want to incorporate 
technology-based tools that support augmented memory, 
if memory is not the construct being tested. Picture a 
wearable, highly personalized computer in a wireless 
environment that quickly searches books and articles the 
student has read, essays she has written, and conversa- 
tions she has participated in on a specific topic. 

Given this extreme for rhetorical purposes, the panel 
unanimously expresses that they are not prepared to 
hand complete control of test formats and use of accom- 
modations to students. Decisions for assessment format 
and accommodations are, they maintain, the province of 
educational teams. Asked if we should allow every 
student the accommodations they deem best, Van Kuren 
replies, “No. The purpose of accommodations is to level 
the playing field, not to give students an extra advantage. 

. . . Even students with disabilities are not given ‘carte 
blanche’ concerning accommodations.” 

The response is motivated primarily by the need to 
balance flexibility with reliability and validity. Flexibility, 
in the panelists’ experiences, tends to be viewed as the 
antithesis of reliability and validity by assessment profes- 
sionals. Accommodations, our best current example of 
flexibility in assessment, certainly call these psychometric 
guideposts into question. Every accommodation, be it a 
different format for the question, a different environment 
to avoid distractions, or a human reader, introduces an 
unknown element for the meaning of the aggregate 
information. On the other hand, every missed or mis- 
matched accommodation, in Sparks’ words, “results in 
scores that do not represent learner knowledge.” 

This complicated dance, like universal design for assess- 
ment itself, is just beginning. Van Kuren, for example, 
reconsiders her response to opening the flood gates of 
accommodations. 

As I was driving home, another thought kept 
niggling. ... As long as we are bound by the 
constraints of standardized assessments as they are 
given today, “accommodations” will need to be 
restricted to students with disabilities to level the 
playing field, as I mentioned before. 
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However, the better system would be one in 
which all students can use their best reporting 
mode when being assessed on their mastery of 
standards. That holds the promise of each student 
truly being able to show their knowledge and the 
skills they have mastered, and of providing 
educators with the information they need to 
provide appropriate instruction to each student. 
This type of system may take us far from standard- 
ized assessments, but that wouldn’t necessarily be a 
bad thing! 



URLS 

Soapbox: www.iaete.org/soapbox/ 

Panelist Affiliations 

Center for Applied Special Technology (CAST): 
www.cast.org 

Consortium for School Networking (CoSN): 
www.cosn.org 

Council for Exceptional Children (CEC): 
www.cec.sped.org 

Virginia Department of Education: www.pen.kl2.va.us 



For More Information 

Universal Design for Learning: A Guide for Teachers and 
Education Professionals by the Council for Exceptional 
Children 

This short book provides a general overview of universal 
design and provides background on federal legislation 
that supports the concept. A brief description of teaching 
strategies that support universal design and examples of 
assistive technologies that support common learning 
disabilities are also described. Of particular interest is the 
explanation of Schumm, Vaughn, and Leavell's "Plan- 
ning Pyramid" as a structure for developing learning 
goals in an inclusive classroom. 

Teaching Every Student in the Digital Age by David H. 
Rose and Anne Meyer with Nicole Strangman and 
Gabrielle Rappolt 

This book from the co-directors of CAST provides a 
thorough explanation of universal design for learning 
beginning with a review of supporting brain research 
that is easy to read and helpful for understanding how 
and why students have differences in learning styles, 
preferences, and motivation. There is specific detail in 
this book about incorporating universal design for 
learning in the classroom and how it can support goal 
setting, instruction, and assessment. There are numerous 
examples and scenarios that help to illustrate universal 
design for learning in several classroom settings with 
diverse instructional challenges. 



The URLs cited are for the reader’s ease of reference. They are not 
intended to be comprehensive, and their inclusion should not be 
interpreted as an endorsement of any corporation’s products or services 
by AEL. All URLs were active at the time of publication. 
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